1. Field of the Invention
The present invention relates to a technique of correcting a portion of an image which is captured by an image capturing apparatus using an image sensor such as a CCD or CMOS sensor, the image quality of which has deteriorated due to a foreign substance adhering to an optical element placed in front of the image sensor.
2. Description of the Related Art
Recently, many image capturing apparatuses have appeared on the market which generate an image signal by using an image sensor such as a CCD, and record it as data, such as for example digital cameras and digital video cameras. A digital camera eliminates the necessity of a photosensitive film, which has been conventionally used as a recording medium, and records an image as data on a data recording medium such as a semiconductor memory card or a hard disk drive instead of such a film. These data recording media allow repetitive write and erase operations unlike films, and hence can reduce the consumable expenditure. Such media are therefore very useful.
In general, a digital camera is equipped with an LCD (Liquid Crystal Display) monitor capable of displaying captured images as needed, and a detachable large-capacity storage device.
Using a digital camera comprising these two devices can eliminate the necessity of a film, which is a recording medium conventionally used as a consumable, and allows the user to immediately check an image on the spot by displaying it on the LCD monitor device. It is therefore possible to erase any unsatisfactory image data on the spot or perform re-shooting as needed. That is, as compared with silver halide cameras using film, digital cameras have remarkably improved the efficiency of taking photographs.
Owing to such convenience and technical innovation such as an increase in the resolution of image sensors, the application range of digital cameras has been expanded. Recently, there have been available many lens interchangeable digital cameras such as single-lens reflex cameras.
In a digital camera, however, small particulates such as dust or motes (to be simply referred to as dust hereinafter) sometimes adhere to an image sensor protective glass fixed to an image sensor or the surface of an optical filter or the like (which will be generically referred to as an image sensor optical component hereinafter) placed in front of an image sensor. When dust adheres to an image sensor optical component in this manner, the dust blocks light. As a result, for example, the corresponding portion of the scene cannot be captured. That is, the quality of the captured image deteriorates.
In cameras using silver halide films as well as in digital cameras, dust on a film is also captured on an image. However, the film moves frame by frame, and hence similar dust is very rarely captured on all frames.
In contrast, the image sensor of the digital camera does not move, and the camera performs shooting using the same image sensor. For this reason, once dust adheres to an image sensor optical component, it is captured on many frames (shot images) in the same manner. A lens interchangeable digital camera, in particular, has a problem that dust tends to enter the camera when lenses are changed.
The user of the camera therefore must always take care about the adhesion of dust to an image sensor optical component, and spend much effort to check for dust and clean it off. Since an image sensor, in particular, is placed relatively deep inside the camera, it is not easy to clean or check for dust.
Dust easily enters a lens interchangeable digital camera when a lens is attached/detached. Furthermore, in many lens interchangeable digital cameras, a focal plane shutter is placed immediately before an image sensor, and hence dust easily adheres to an image sensor optical component.
Since such dust on an image sensor optical component generally adheres onto a protective glass or an optical filter instead of the surface of an image sensor, the dust is imaged in different states depending on the aperture value of a shooting lens or the distance from the pupil position of the lens. That is, as the aperture value approaches the open F-number, a shot dust image is blurred, and hence even the adhesion of small dust has almost no influence on the image. In contrast, as the aperture value increases, such dust is clearly imaged, and hence affects the image.
To solve this problem, for example, a user uses an image editing program to read an image in which dust is captured, designate an area including the dust, determine the presence/absence of the dust in the designated area, and execute interpolation processing (to be referred to as a repair process hereinafter), thereby removing the dust portion.
For the area designated by the user, however, the repair process described above executes no interpolation processing when no dust is detected or when dust is in contact with the designated area. This is because executing interpolation without appropriately detecting the dust area often results in an unnatural image. In this case, no processing is performed even if the user repeatedly designates the same area for repair.
Also, there is known a method of making dust less noticeable by preparing in advance an image of only dust on an image sensor, which is obtained by shooting a white wall or the like while the lens is set at a large aperture value, and using the image in combination with a general shot image (see Japanese Patent Laid-Open No. 2004-222231). This method, however, is cumbersome because the user must always be conscious of the correspondence between the image shot for dust detection and shot images to be associated with it. It is therefore conceivable to hold information indicating the position of dust, which is acquired by shooting a white wall or the like, and attaching the information indicating the position or size of dust to image data obtained by general shooting operation. For example, there is a method (automatic repair process) which uses a separately prepared image processing apparatus to analyze the dust position on the image data from the attached information of the position of dust and make the image of the dust less noticeable by interpolating the analyzed area with surrounding pixels.
In the automatic repair process described above, however, no interpolation processing of the automatic repair process is executed when it is determined that no dust exists or when the dust is in contact with the designated area. This is because, as in the repair process, executing interpolation without appropriately detecting the dust area often results in an unnatural image. That is, in this case, the automatic repair process cannot be performed for that area.